Coaching Web-based Collaborative Learning based on Problem Solution Differences and Participation

This paper describes the design and evaluation of a coach that helps students collaborate while solving Entity Relationship modeling problems in a computer-mediated learning environment (COLER). Unlike previous work generally emphasizing dialogue analysis or expert models, this work evaluates a new approach to supporting collaboration that identifies learning opportunities based on differences between problem solutions and tracking levels of participation. The contribution made by these and other knowledge sources in the generation of collaboration advice was evaluated by comparing expert rankings of advice to the software coach's rankings, and by identifying the advice that would be lost if each respective knowledge source were removed. Results show that good quality advice can be obtained through these knowledge sources, although other knowledge sources may fill in gaps relative to the expert's performance. This work demonstrates how intelligent agents can produce reasonable collaboration advice in domains for which structured problem solutions exist by using a few basic knowledge sources, and illustrates several methods of evaluating the knowledge and reasoning of complex knowledge-based systems.